fix: rebuild index when mode adds capabilities - #1263

Summary

Fixes #1273
Related: #563

• persist the effective index capability level in the Project node as index_mode
• force a full rebuild when the requested mode requires capabilities missing from the stored index
• preserve additive data on downgrades by separating requested discovery mode from stored effective extraction mode
• migrate legacy, malformed, or non-exact mode metadata with one conservative rebuild
• keep existing persistent artifacts synchronized after any reindex, refreshing at FAST quality when persistence is not explicitly requested

Root cause

Incremental routing originally considered stored file hashes and the discovered-file threshold, but not the capability level of the stored index. This caused unchanged fast → moderate upgrades to take the incremental no-op path.

Changed-file downgrades re-extracted with the weaker requested mode, which could remove stored capabilities such as SIMILAR_TO edges or full-mode Macro nodes.

The metadata parser also compared yyjson strings with strcmp(). A valid JSON string containing "full" + U+0000 + "garbage" therefore matched "full" at the embedded NUL and could incorrectly select incremental/no-op instead of the conservative rebuild route.

Mode transitions

Downgrades keep the stored index_mode; p->mode and Project metadata are not weakened. Files outside requested discovery continue to use the existing mode-skipped/hash mechanism. A no-change run leaves the on-disk DB untouched. If a persistent artifact already exists, any reindex refreshes it after publication — at FAST quality unless persistence is explicitly requested, so the artifact encoding no longer oscillates with the route and full rebuilds no longer pay zstd-9 + double VACUUM by default.

The public MCP API and SQLite schema are unchanged. Existing changed-file threshold and ADR preservation behavior are unchanged.

Regression coverage

• unchanged fast → moderate: fail-before incremental no-op left SIMILAR_TO at 0; pass-after rebuild creates similarity data and records index_mode=moderate
• changed-file moderate → fast: preserves SIMILAR_TO and index_mode=moderate; the following moderate no-op remains stable
• changed-file full → moderate: preserves full-only Macro nodes and index_mode=full; the following full no-op remains stable
• legacy Project metadata {} triggers a rebuild and restores index_mode
• an escaped-NUL index_mode ("full\^@garbage") is rejected by exact yyjson_equals_str comparison, rebuilt once, and normalized to the requested mode
• explicit persistence failure on a changed-file downgrade propagates as a run failure
• no-op downgrade with explicit persistence still produces the artifact via post-publish export

Verification

PR head is 902262e7 (single DCO commit, rebased onto current main).

• fail-before downgrade suite: 222 passed, 2 failed
• fail-before escaped-NUL regression (strcmp substituted for yyjson_equals_str): 231 passed, 1 failed — the new test pins the exact-comparison requirement
• ASAN_OPTIONS=detect_leaks=0 build/c/test-runner pipeline — 232 passed
• make -f Makefile.cbm lint-format lint-no-suppress — passed
• git diff --check against main — passed
• scripts/test.sh locally: 6741 passed, 2 failed, 4 skipped (121 suites) — both failures are local macOS environment stalls (dyld hanging while exec'ing copied runtime images from /tmp, before any repo code runs; neither daemon_runtime nor index_supervisor is touched by this diff)
• GitHub CI on the final head: fully green — all test suites (unix shards, tsan, windows-guards), pr-smoke on ubuntu/macos-14/windows, lint, security, DCO, CodeQL (run 30150702836)

Review follow-up: the PR was slimmed from 1016 to 588 added lines (−42%) after a deep review pass. Removed periphery with no contract change: the byte-exact properties read API, the no-change downgrade coverage carry-forward, and the changed-file linear-scan fallback; metadata reads now go through the existing cbm_store_find_node_by_qn.